Characterisation of paper

ABSTRACT

A method and system for characterising paper, where from images of numerous paper samples are extracted multi-dimensional features describing features of the paper; the said features are entered as input into a learning classifier operating in an unsupervised manner, which produces an projection of the said data of each picture part in a low-dimension space in such a way that paper grades having close properties produce close projection in the low-dimension space and the classification results depicted in the low-dimension space are used to aid classification.

The invention relates to the characterisation and classification ofpaper quality by using computer vision or other two-dimensionallydescriptive method.

To the application is appended a bibliography, which is referred to byreference numerals in square brackets. Prior art is referred to in theform of cited references in connection with the aspect at hand,respectively.

The aim of the invention is to accomplish a method for thecharacterisation of paper quality that will provide more reliableclassification than current methods, without variation due to humanfactors.

Paper grading systems based on computer vision—which represent the priorart—were previously founded on supervised learning methods and old andinefficient features computed from images. As features have usually beenused measurements obtained from co-occurrence matrices, power spectrumanalysis and the specific perimeter feature. Also, the average of thegrey shades and variance of the images have been presumed to representvariations in paper grammage. Of the features has been formed anumerical quantity, which describes the quality of paper. On the basisof this numerical quantity, the formation or other properties of thepaper have then been classified. [1, 2, 3, 4, 5]

The old textural features are unable to provide very accurateinformation on paper texture and they are sensitive to changes inconditions, such as lighting. When poorly discriminating features arecombined with supervised training of a classifier, the characterisationcapacity of the system is further impaired. This is due to the fact thatthe conventional supervised methods are extremely sensitive to humanerrors. People usually make errors in selecting the training samples andin naming them. In addition, the selections made by humans aresubjective and thus the interpretations of different people differ fromone another. From the point of view of quality inspection this isundesirable. Re-training a system based on supervised learning methodsis difficult, should the changes in conditions so require. This is oftenthe case, because less developed textural features are extremelysensitive to changes in the conditions.

A problem has been that paper has been analysed with poorlydiscriminating textural features. Furthermore, attempts have been madeto specify class boundaries in an already fragmented and non-normallydistributed feature space by means of parametric methods. Supervisedmethods have been used in training the classifiers and in seeking theclass boundaries, which increases the amount of errors.

In characterising paper, the aim is to classify papers sharing the sameproperties in the same category. Paper may be imaged throughout itsmanufacture, which will also give information on the properties of goodor poor paper during the different stages of manufacture. Withoutcharacterisation, on the basis of images alone, it is not possible toseek useful information on the process, because the assessment andclassification of images is very difficult for man as well as beingsubjective and, in addition, processing a large amount of data withoutautomatic classification based on numerical values or symbols isimpossible. By means of characterisation, the quality of paper can beclassified into several classes on the basis of which the operation ofthe manufacturing process can be traced and attempts can be made toimprove certain properties of the paper, so long as it is known whichfactors affect the quality of paper, and what the paper has been like ateach stage of manufacture, respectively. Characterisation itself doesnot have to take a stand on the quality of the paper, it suffices thatsimilar papers are classified into the same class. The process may becontrolled or the paper can be classified into quality classes inaccordance with the classification.

In computer vision methods, the aim is to calculate a number offeatures, which will describe the properties of paper as accurately aspossible [1, 2, 3, 4, 5]. Typical properties are, for example, theprintability and tensile strength of the paper. The features calculatedare numerical quantities and they form clusters fragmented in amulti-dimensional feature space. The feature space may be extremelymulti-dimensional, and it is obvious that the features describingdifferent paper grades are difficult to find in the fragmented space.FIG. 1 shows an example of a feature space presented, for the sake ofsimplicity, in a two-dimensional system of coordinates. The crosses inthe Figure represent the values of the features, and the line drawn inthe Figure the possible change in the printability properties of thepaper.

The specification refers to the following Figures:

FIG. 1 shows the fragmentation of features and the boundary ofproperties.

FIG. 2 shows the clustering of multi-dimensional feature data in atwo-dimensional system of coordinates.

FIG. 3 shows a diagram in principle of classification according to theinvention.

FIG. 4 shows the calculation of a 3×3 size LBP feature.

FIG. 5 shows the neighbourhood of a point on the circumference fromwhich the LBP feature is calculated.

FIG. 6 shows the use of a SOM as a classifier.

FIG. 7 shows a diagrammatic view of paper characterisation duringmanufacture.

Conventional parametric methods are unable to find the boundariesbetween different paper grades accurately, because they make assumptionson the distribution of data. In the method according to the invention,the data is first depicted in a two-dimensional system of coordinates.Each cluster is given a label on the basis of the type of paper thecluster represents. In other words, deductions on the quality of thepaper can be made on the basis of the location of the sample in thetwo-dimensional system of coordinates. FIG. 2 shows an example ofdescribing a multi-dimensional feature space in a two-dimensional systemof coordinates by means of a method, which maintains the local structureof the data and the mutual distances between samples [6, 7, 8, 9, 10].Labels 3 a-3 d represent different properties of the paper; paperclassified in an area marked by the same label is similar to otherpapers in the same class with respect to the property in question. Thelabels are given afterwards and, for example, tensile strength, degreeof gloss or printability are usually divided into different regions andobviously have different labels.

In the method, the data is organised automatically in such a way thatthe mutual locations of the samples in the new system of coordinates arethe same as in the original multi-dimensional feature space. Reliabledeductions on paper grades can be made on the basis of where they arelocated in the new system of coordinates. At first, no deductionswhatsoever are made on the distribution of the data, and it may be ofany kind. Papers having different textures may still have similar printproperties. This may be taken into account when labelling the differentclusters. With efficient textural features, such as LBP, the surfacetexture of paper can be analysed extremely efficiently [11, 12].

In the present invention, an unsupervised learning method, efficientgrey-shade variant textural features and illustrative visualisation ofmulti-dimensional feature data are combined by reducing the dimensionsof the feature space. In the method, human assumptions and deductions donot need to be made concerning the training material, but the trainingdata will be organised automatically in accordance with its properties.The multi-dimensional feature space is depicted in an illustrative formand the location of the samples in the feature space can be visualised.

New, sophisticated texture methods give precise information on themicrostructure of the texture. Such grey-shade invariant texturalfeatures are, for example, LBP features, which measure local binarypatterns, and its modifications [11, 12]. When the surface of paper isexamined using these features, important properties of the paper may bediscovered. By combining efficient textural features with anunsupervised learning method, the accuracy of grading can be greatlyimproved.

A diagrammatic view of the method is shown in FIG. 3. From the trainingset 11 are first calculated textural features at stage 12, which arethen used to train the classifier. The dimensions of themulti-dimensional feature space are reduced in order that it can beillustratively visualised. Classification is also carried out by using anew feature space 14. The task remaining to man is to name and selectclassified areas and, at the next stage, to render them into a moreeasily understandable form or to place the paper grades in an order ofsuperiority, so that the process may subsequently be regulated on thebasis of them. It is also a task for man to select the training set insuch a way that a representative sample of different papers is obtained.These tasks are indicated by reference numerals 15, 16, 17 and 18.

In the method, the properties of paper are first described by means ofefficient textural features, which reduces the fragmentation of thefeature space markedly. A multi-dimensional feature space is depicted ina low-dimension system of coordinates in such a way that the localstructure of the data is preserved. The clusters in the low-dimensionsystem of coordinates represent different paper grades. The differentclusters are named in accordance with the paper grade represented by thecluster in question. After this, in the new system of coordinates can beclassified different grades of paper by finding the cluster to which thepaper being examined is clustered. A diagram representing a clusteredfeature space is shown in FIG. 2.

The features may be extracted, for example, by using textural quantitiesbased on local binary patterns. LBP (Local Binary Pattern) featuresdescribe patterns appearing in a local image-level environment [11, 12].An original LBP feature [11] is, for example, a textural featurecalculated from a 3×3 environment, the calculation of which isillustrated in FIG. 4. In the example shown in the Figure, the 3×3environment 31 is categorised by threshold values (arrow 41) inaccordance with the grey shade of the centre point (CV) of theenvironment so as to have two levels 32: pixels greater than or equal tothe threshold value CV are given the value 1, and lower values obtainthe threshold value 0. Subsequent to categorisation by threshold values,the values 32 obtained are multiplied (arrow 42) by an LBP operator 33,which gives an input matrix 34, the elements in which are added up(arrow 44), which gives the value of the LBP. Another way of conceivingthe calculation of the LBP is to form an 8-bit code word directly fromthe threshold value environment. In the case of the example, the codeword would be 10010101₂, which is 149 in the decimal system.

Of LBP features have also been created various multi-resolution androtation invariant methods [12]. In addition, the effect of differentbinary patterns on the performance of the LBP operator have beenexamined, whereby it has been made possible to omit certain patterns informing the feature distribution [12]. In this way it has been possibleto shorten the LBP feature distribution.

Multi-resolution LBP means that the neighbourhood of the point has beenselected from several different distances. The distance may in principlebe any positive number, and the number of points used in the calculationmay also vary according to distance. FIG. 5 shows the neighbourhood of apoint at a distance of four (d=4). Around the point is drawn a circle,the radius of which is equal to the distance selected. From thecircumference are selected samples at distances indicated by the angle αin such a way that Nα=2π, where N is the number of selected samples. Ifa sample on the circumference does not match a pixel accurately, it isinterpolated, by means of which the coordinates of the point are made tocorrespond to the coordinates on the circumference. Distances typicallyused are 1, 2 and 3, and the numbers of samples are correspondingly 8,16 and 24. The more points are selected, the greater the LBPdistribution obtained. A 24-dimensional feature space produces a LBPdistribution containing over 16 million poles.

Using extensive LBP distributions in calculation is cumbersome. The sizeof the distribution can be reduced to a more reasonable size forcalculation by taking into account only a certain, pre-selected part ofthe LBP codes. The selected codes are so-called continuous binary codesin which the numbers on the circumference include at most two bitexchanges from 0 to 1 or vice versa. Thus the code words selectedcontain long, continuous chains comprised of zeros and ones. Theselection of the codes is based on the knowledge that by means ofcertain LBP patterns can be expressed as much as over 90% of thepatterning in the texture. By using only these continuous binary chainsin calculation, an LBP distribution of 8 samples can be reduced from 256to 58. An LBP distribution with 16 samples is, on the other hand,reduced from over 65 thousand to 242, and a distribution of 24 samplesfrom over 16 million to 554 [12].

In the calculation of the LBP feature of a rotation invariant isincluded a pre-selected subset of LBP patterns [12]. The patterns havebeen selected in such a way that they are invariant to rotation takingplace in the texture. Using the LBP features of rotation invariants in anon-invariant problem reduces the capacity of the feature. Thecharacterisation of paper is not, however, a rotation invariant problem.

Classification and clustering may be carried out, for example, byapplying techniques based on self-organising maps [13]. Aself-organising map, the SOM, is a method of unsupervised learning basedon artificial neural networks. The SOM makes possible the presentationof multi-dimensional data to man in a more illustrative, usuallytwo-dimensional form.

A SOM aims to present data in such a way that the distances betweensamples in the new two-dimensional system of coordinates will correspondas accurately as possible to the distances between the real samples intheir original system of coordinates. The SOM does not aim to separatelysearch the data for the clusters it may contain or to display them, butinstead presents an estimate of the probability density of data asreliably as possible, while maintaining its local structure. This meansthat if the two-dimensional map shows dense clusters formed by samples,then these samples are located close to one another in the feature spacealso in reality [13].

In order that the SOM can be used to group a certain type of data, itmust first be trained. The SOM is trained by means of an iterative,unsupervised method [13]. Following the training of the SOM, there is apoint set in the multi-dimensional space for each node on the map, towhich the node corresponds. An algorithm has adjusted the map by meansof training samples. Multi-dimensional vectors form a non-linearprojection in the two-dimensional system of coordinates, thus makingclear visualisation of the clusters possible [13].

The use of the SOM as a classifier is based on the clustering of similarsamples close to one another, which means that they can be defined astheir own classes on the map. The samples of nodes far from each otherare mutually different, whereby they can be distinguished to belong todifferent classes. FIG. 6 shows the clustering of good and poor paper inopposite corners of the map. FIG. 6 shows the use of the SOM as aclassifier. Samples 61, 62 in the Figure are classified in classes 63,64. As a rough example has been shown the classification of good paper61 in class area 63, and the classification of poor paper in area 64. Itshould be noted that there may be several areas of both good and poorpaper fragmented in different parts of, for example, a two-dimensionalspace, but in such a way, however, that for example all paper classifiedin area 64 is poor in the same respect. It is understandable, that it isvery useful for the paper manufacturer to know which conditions producepaper of the said kind, so that the conditions producing poor qualitycan be avoided in manufacture. This is possible by monitoring theproduction parameters and by continuously classifying the quality ofpaper, whereby new aspects will be learnt of the operation of theprocess. It is also possible to enter the process parameters and theresults of paper classification into another SOM classifier, whereby asystem learning from errors is obtained, which can be used as an aid inprocess control. This will give as a final outcome a classificationwhich describes the conditions of manufacture with respect to thequality of paper. The system thus learns, for example the effect ofhundreds of variables on paper quality.

Above is described classification according to the invention using SOMclassification, but any unsupervised clustering method is suitable foruse in the classification according to the invention, for example, theLLE, ISOMAP and GTM techniques which are not actual neural networktechniques.

The method is suitable for use in the quality inspection of paper duringpaper manufacture, for example, as shown in diagram 7. Pictures aretaken with a fast camera of the moving paper web 74 in connection withthe paper machine 75. The diagram in the Figure shows a background light73; depending on the need also, for example, a diagonal front light canbe used. After this, deductions on the qualitative properties of thepaper being produced can be made, and the any adjustments in theprogressing of the process may be carried out. The method beingpresented here would be used in connection with the computer 71 shown inthe Figure. Rapid image analysis and an illustrative user interface forextensive measurement data provide an enormous amount of additionalinformation on the paper being produced to the paper manufacturersthemselves.

Features are extracted from the pictures taken during the image analysisby means of the techniques mentioned above, and classification intodifferent quality classes is carried out. By means of the userinterface, the progressing of the quality of the paper can be followedas production progresses.

By means of the method, paper can be analysed almost throughout itsproduction cycle. The power of the background light must, however, beincreased if pictures are taken of already coated paper. In addition,the capacity of textural features may be impaired with coated papers.

Exact information on the quality of paper during its productionfacilitates studies carried out by the paper manufacturer. An automationmanufacturer may integrate the system to be a part of the overallprocess and its adjustment.

The invention is characterised by what is presented in the independentclaims and the dependent claims describe its preferred embodiments.

APPENDIX: BIBLIOGRAPHY

-   [1] Cresson T. M., Tomimasu H. & Luner P. (1990) Characterization of    Paper Formation, Part 1: Sensing Paper Formation. Tappi Journal:    Vol. 73, No. 7: p. 153-159.-   [2] Cresson T. & Luner P. (1990) Characterization of Paper    Formation, Part 2: The Texture Analysis of Paper Formation. Tappi    Journal: Vol. 73, No. 12: p. 175-184.-   [2] Cresson T. & Luner P. (1991) Characterization of Paper    Formation, Part 3: The Use of Texture Maps to Describe Paper    Formation. Tappi Journal: Vol. 74, No. 2: p. 167-175.-   [3] Sudhakara R. P., Stridhar R., Gopal A., Meenakshi K., Revathy    R., Chitra K. & Palaniandi D. (2001) Optical Paper Formation    Analyzer. CEERI Centre, India.-   [4] Bernie J. P. & Douglas W. J. M. (1996) Local Grammage    Distribution and Formation of Paper by Light Transmission Image    Analysis. Tappi Journal: Vol. 79, No. 1: p. 193-202.-   [5] Bouyndain M., Colom J. F., Navarro R. & Pladellorens J. (2001)    Determination of Paper Formation by Fourier Analysis of Light    Transmission Images. Appita Journal: Vol. 54, No. 2: p. 103-105,    115.-   [6] Kohonen T. (1997) Self-organizing Maps. Springer-Verlag, Berlin,    Saksa, 426 p.-   [7] Roweis S. T. & Saul L. K. (2000) Nonlinear Dimensionality    Reduction by Locally Linear Embedding. Science Magazine, Vol 290, 22    Dec. 2000: p. 2323-2326.-   [8] Roweis S. T. & Saul L. K. (2001) An Introduction to Locally    Linear Embedding. URL:    http://www.cs.toronto.edu/˜roweis/lle/papers/lleintroa4.pdf    (13.5.2002).-   [9] Svensên J. F. M. (1998) GTM: The Generative Topographic Mapping.    Doctoral thesis. Aston University, Englanti, 108 p.-   [10] Tenenbaum J. B. (1998) Mapping a Manifold of Perceptual    Observations. Advances in Neural Information Processing Systems,    Vol. 10.-   [11] Ojala T., Pietikäinen M. & Harwood D. (1996) A Comparative    Study of Texture Measures With Classification Based on Feature    Distributions. Pattern Recognition, Vol. 29, No. 1, p. 51-59.-   [12] Ojala T., Pietikäinen M. & Mäenpää T. (2002) Multiresolution    Gray-Scale and Rotation Invariant Texture Classification with Local    Binary Patterns. IEEE Transactions on Pattern Analysis and Machine    Intelligence, Vol. 24, No. 7.-   [13] Kohonen T. (1997) Self-organizing Maps. Springer-Verlag,    Berlin, Saksa, 426 p.

1. A method for characterising features of paper based on computervision, characterised in that from pictures of numerous paper samplesare extracted multi-dimensional features describing features of paper;the said features are entered as input into a learning classifieroperating in an unsupervised manner, which produces a projection of thesaid data of each picture part in a low-dimension space, so that papergrades having close properties produce close projections in thelow-dimension space and the classification results projected in thelow-dimension space are used to aid classification.
 2. A method forcharacterising paper as claimed in claim 1, characterised in that thesaid learning system operating in an unsupervised manner is anunsupervised clustering method or its simulation, for example, a SOM(Self-Organising Map).
 3. A method for characterising paper as claimedin claim 1, characterised in that the feature describing the papersamples is a LBP or a bit pattern feature derived from it.
 4. A methodfor characterising features of paper as claimed in claim 1,characterised in that according to the method, paper is in additionimaged and classified at different stages of its manufacture.
 5. Amethod for characterising features of paper as claimed in claim 4,characterised in that the samples imaged at different stages of themanufacture are processed further by means of the unsupervised learningclassifier in such a way that the classification will also concern theprogressing of the manufacturing process.
 6. A method as claimed inclaim 5, characterised in that in addition to the image information,selected process parameters and/or measurement results are used asinput.
 7. A system for classifying paper using computer vision,characterised in that the system comprises imaging means, means forextracting the features describing paper quality from an image of thepaper, and means for unsupervised learning classification into a spacewith a low-dimension space compared with the feature space.